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Abstract 

Federated learning (FL) enables privacy-preserving 
collaborative model training. When businesses 
with potentially competing interests join FL, it 
is crucial to compensate them for their economic 
costs incurred in order to sustain participation. At 
the same time, the FL coordinator needs to be 
selective in order to retain participants who con¬ 
tribute positively to the federation. Two types of 
information asymmetry in FL hinders these ob¬ 
jectives: 1) the potential usefulness of partici¬ 
pants’ original datasets and 2) private costs incurred 
by FL participants. In this paper, we propose a 
Vickrey-Clarke-Groves-based FL incentive mech¬ 
anism, FVCG, to address these problems by rec¬ 
ommending the FL coordinator to selectively re¬ 
ject costly data based on ex-ante evaluations of 
the usefulness of the offered data. Such a right- 
of-rejection encourages data owners to offer their 
original high-quality datasets and truthfully report 
their cost types. FVCG provides theoretical guar¬ 
antees for incentive compatibility and social sur¬ 
plus maximization. It minimizes expected devia¬ 
tions from individual rationality, is by expectation 
weakly budget balanced, and supports user-defined 
fairness criteria. 


1 Introduction 

Data is key to building artificial intelligence (AI) empow¬ 
ered applications. However, in practice, data are collected 
and stored by different entities which are either unwilling or 
unable to share data directly with others [Yang et al, 2019a; 
Kairouz et al, 2019; Yang et al, 2019b]. Federated learning 
(FL) has emerged in recent years as an alternative solution 
to build AI models based on distributedly stored data while 
preserving data privacy. 
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Figure 1: The flow of incentives in federated learning 


Nevertheless, FL may not always be profitable for partici¬ 
pants. This is especially the case for businesses participating 
in FL. Besides the cost of gathering and cleaning data, busi¬ 
ness participants also bear technical, compliance and oppor¬ 
tunity costs (e.g., due to their market share erosion) [Yu et al, 
2020]. In addition, there exists the problem of free riding. If 
every participant enjoys the benefit of the FL model equally, 
a rational participant may only join FL training with its low- 
quality data and conceal high-quality data so as to protect its 
competitive advantage. Without a properly designed incen¬ 
tive scheme, the aforementioned problems will threaten the 
sustainability of a federation over time. 

In FL, model parameters are updated based on locally 
stored datasets from participants (i.e., agents) in an encrypted 
manner. Although no raw data are being physically trans¬ 
mitted from the participants to the federation, for simplicity 
of terms, we refer to “joining federated model training” and 
“contributing data” interchangeably in this paper. We propose 
the design of an incentive scheme as illustrated by Figure 1 to 
incentivize participants. As shown by the figure, revenue gen¬ 
erated by the FL model is distributed among participants to 
cover their costs of contributing data. Here, there is asymmet¬ 
ric information about both the truthfulness of the contributed 
dataset and the cost. Similar problems in data collaboration 



















have been discussed by [Fallings and Radanovic, 2017]. An 
FL incentive scheme needs to incentivize participants to offer 
their original high-quality datasets and truthfully report their 
private cost types. 

In order to promote truthfulness of participants in FL 
model training, we propose an incentive scheme inspired by 
buyer’s right of rejection. A buyer has the right to reject de¬ 
fective goods. Such an approach transfers the burden of en¬ 
suring the quality of goods from the buyer to the seller. In FL, 
goods are data involved in FL model training; the sellers are 
the participants; the buyer is the FL coordinator. 

In particular, we propose FVCG, a Federated VCG (Vick- 
rey-Clarke-Groves as proposed by [Vickrey, 1961; Clarke, 
1971; Groves, 1973]) incentive scheme, which entitles the FL 
coordinator with the right to reject high-cost-low-quality data. 
Under FVCG, the FL coordinator firstly evaluates the useful¬ 
ness of a participant’s offered data in a sandbox and analyzes 
it against its reported cost to determine if strategic rejection 
shall be triggered. Once the participant passes this round of 
evaluation, FVCG computes the payoff for this participant, 
which consists of the VCG payment component (which guar¬ 
antees incentive compatibility and social surplus maximiza¬ 
tion) and an adjustment payment component (which promotes 
individual rationality, weak budget balancedness, and accom¬ 
modates user-defined fairness criteria). As the adjustment 
payment may not be in a closed form, we propose a neural- 
network-based learning algorithm to compute its value. 

Theoretical analysis proves that, under some reasonable 
conditions, FVCG simultaneously maximizes social surplus, 
minimizes expected deviations from individual rationality, is 
by expectation weakly budget balanced, and supports user- 
defined fairness. Extensive experimental evaluation shows 
that compared to state-of-the-art approaches, FVCG effec¬ 
tively incentivizes participants to offer their original datasets, 
keeps all participants in the federation, results in higher social 
surplus, and brings profits to the FL coordinator. 

2 Related Works 

In recent years, there has been a growing body of literature on 
training machine learning models with strategic participants. 
This line of research can be divided into three groups. 

The first group assumes that predictions from the trained 
model affect participants’ utilities, and hence they may report 
false data to drive model predictions closer to their expec¬ 
tations [Perote and Perote-Pena, 2004; Dekel et al, 2010; 
Caragiannis et al, 2016; Meir et al, 2012]. The second 
group focuses on the loss of privacy and compensates data 
providers for such privacy losses [Ghosh and Roth, 2015; 
Cummings et al., 2015; Nissim et al., 2012; Fleischer and 
Lyu, 2012]. The third group views data as economic re¬ 
sources with both values and costs. The model builder pays 
monetary incentives to encourage data providers to contribute 
higher-quality data with higher values. The value of a dataset 
can be measured by its influence on the performance of the 
trained model (e.g., its Shapley value) [Jia et al, 2019; Wang, 
2019; Cai et al, 2015], with considerations on the costs in¬ 
curred [Richardson et al, 2019; Westenbroek et al., 2019; 
Yu et al., 2020]. 


FVCG belongs to the third group. Compared to previ¬ 
ous works, FVCG is unique as it focuses on FL scenar¬ 
ios involving business participants which are assumed to be 
more strategic and thus, presents a more complex game- 
theoretic environment. Methodologies in these papers are 
adopted from Mechanism Design, an important subfield of 
game theory [Nisan et al., 2007; Koutsoupias, 2014]. FVCG 
is based on the famous VCG mechanism [Vickrey, 1961; 
Clarke, 1971; Groves, 1973] and the literature on the com¬ 
putational aspect of VCG mechanisms [Nisan and Ronen, 
2007]. However, unlike the classical VCG mechanism, 
FVCG can achieve individual rationality and weak budget 
balancedness for rich type spaces under certain conditions, 
releasing the tension between these objectives explained in 
[Jackson, 2014]. 

3 Preliminaries 

We consider a problem with a set of n participants denoted 
by W = {0,1,. .., n — 1}. Each participant i G N owns 
a private dataset. Each participant contributes to the feder¬ 
ated model quality Q with parameter-updates based on the 
whole or part of his original dataset. The federated model 
can generate revenue R{Q), which is to be distributed to the 
participants. Each participant receives a payment p^. 

A dataset is measured by its usefulness, which represents 
how useful the dataset is in updating a given EL model. The 
usefulness of the original private dataset and the contributed 
dataset are denoted by di and di, respectively, di and di 
can be vectors composing of multiple measures of useful¬ 
ness. Since a participant may contribute only a subset of 
the original dataset, we have di < di, where > denotes the 
element-wise vector comparison. We also set the usefulness 
of an empty dataset to be 0, and the original dataset satisfies 
di > 0. The federated model quality Q is assumed to be 
a function of d = {do,..., dn-i). The federation revenue 
R, which depends on Q, also depends on d. We use B{-) to 
denote the composite function, i.e., B{d) = R{Q{d)). 

Joining EL model training incurs costs to participants. It is 
reasonable to assume i’s cost Ci to increase with di. Since the 
cost of contributing a dataset with the same usefulness may 
vary among participants, we introduce another parameter 7 ^, 
called cost type such that q = Ci{di, yf). In our mechanism, 
participants report their private cost types to the EL coordi¬ 
nator when offering their datasets. The reported cost type 
of i is denoted by 7 ^. The participant’s preference is repre¬ 
sented by the quasi-linear utility Ui = pi — Ci. Since we fo¬ 
cus on dominant strategy mechanism design, there is no need 
to impose any assumptions on either the prior distributions 
of participants’ types {di,ji), or participants’ knowledge of 
each other’s type distribution. Nevertheless, lacking real be¬ 
havioral data of participants in practice, we may require such 
prior distributions to generate synthetic data for the training 
of neural networks. 

A federation desires to maximize the social surplus 
S{d, 7 ) = B{d) — Ci{di, 7 i), which is a classical eco¬ 
nomic measure defined as the total benefit minus the total cost 
in an economic system. Social surplus maximization implies 



Pareto efficiency. As the social surplus depends on the use¬ 
fulness of contributed datasets, the FL coordinator cannot op¬ 
timize it without the right-of-rejection. In FVCG, the FL co¬ 
ordinator can reject a participant fully or partially in a given 
round of FL model training. The usefulness of the offered 
dataset by participant i is denoted by di. The usefulness of 
the accepted/contributed dataset is no greater than the useful¬ 
ness of the offered dataset (i.e., di < di). These two use¬ 
fulness measures are related by rji = di (Z) di G [0, 
and di = di Q rji, where © and 0 denote the element-wise 
multiplication and division, respectively, rji is the acceptance 
level, which is a continuous value in the (iim(fii)-dimensional 
cube [0,1]'^®™*^'^*^. The FL coordinator controls the accep¬ 
tance level. For example, pi = 0 means that the offered 
dataset is entirely rejected, whereas rji = 1 means that the 
offered dataset is entirely accepted. 0 < < 1 means the 

offered dataset is partially accepted. 

In order for the FL coordinator to evaluate the usefulness 
of a participant’s offered dataset before accepting it, we as¬ 
sume the existence of a sandbox. A sandbox is a testing en¬ 
vironment isolated from the production environment. A copy 
of the current version of the FL model can be placed in the 
sandbox, based on which participants can evaluate the use¬ 
fulness of their datasets through algorithms such as [Koh and 
Liang, 2017]. As the design of the sandbox is not the focus 
of this paper, we assume its availability and FVCG can obtain 
the usefulness of a given participant’s offered dataset from it. 


4 The Proposed FVCG Approach 

The FVCG payment pi to participant i is composed of the 
VCG payment Ti and the adjustment payment h* + g*. FVCG 
computes these payments through the following procedure. 

4.1 Computing the Optimal Acceptance Levels 


4.2 Computing the VCG Payments 

With T]*, the VCG payment Ti to participant i can be com¬ 
puted as: 

Ti = S*{d,^) - Sl,{d_i,^_i) + Ci{d^ Q p* 

= [B{dQr]*{d,^)) - B{d_i Q {d_i,j_i))] (2) 

- '^[ck{dk&vl{d,j),yk) -Cfc(4 ©??fc”(d_,,7_i),7fc)], 

k^i 

where denotes the offered data usefulness and 

reported cost types excluding participant i, and ri~^* and 
S*_i{d-i, 7 _i) are the corresponding optimal data acceptance 
levels and social surplus, r is a function of (d, 7 ), written as 
T(d, 7 ). Note that is different from rylj. The former 
maximizes S{d-i 0 rj-i, 7 _i), whereas the latter is the com¬ 
ponent of rj* that maximizes S'(<i © ry, 7 ). 

4.3 Learning the Optimal Adjustment Payments 

The adjustment payments are introduced to promote individ¬ 
ual rationality, weak budget balancedness, and user-defined 
fairness criteria, without hurting incentive compatibility and 
social surplus maximization guaranteed by the VCG pay¬ 
ments. We set the adjustment payment to participant i as: 

di{d—iy'y—i) gii.di), (3) 

where hi{ ) is a function of (<l_j, 7 _i), and pi(-) is an in¬ 
creasing function of di. 

As in the standard VCG mechanism, adding the adjustment 
payment hi{-) in Eq. 3 to the VCG payment does not af¬ 
fect incentive compatibility and social surplus maximization. 
But different from the standard VCG mechanism, we also add 
Pi (•), which may affect incentive compatibility. So we require 
gi{-) to be increasing to guarantee that offering the original 
dataset with usefulness di is incentive compatible. The op¬ 
timal adjustment payment functions and p*(-) are such 
chosen that the total payment to participant i, calculated by 


To determine the optimal level of acceptance for each par¬ 
ticipant’s local dataset, we optimize the following objective 
function: 

Tj ^ argmax^^jQ 0^7,7)} ( 1 ) 

n—1 

= argmax^g[Q^;^]dim(di)x™{i7(d© 

where r}* = (ryj,..., Pn-i)- Different (d, 7 ) results in dif¬ 
ferent jy*. Hence, ry* is written as ri*{d, 7 ). 

The maximum social surplus is denoted by S*{d,'y) = 
B(d 0 jy*(d, 7 )) - YllZo © vt Although 

S*{d,j) and S{d,^) both represent social surplus, they are 
different functions. The independent variables of S{-) are the 
usefulness of the contributed dataset d and the true cost types 
7 , whereas the independent variables of S*{-) are the useful¬ 
ness of the offered dataset d and the reported cost types 7 . 


Pi{') — Tii') + h*{-) + g*{-), (4) 

minimizes the deviations from individual rationality (i.e., 
Lossl), weak budget balancedness (i.e., Loss2), and user de¬ 
fined objectives such as fairness (i.e., Loss3) in the following 
loss function: 


LOSS = AiLossl -I- A2L0SS2 -I- A3L0SS3. (5) 


The expressions of Lossl and Loss2 will be provided in the 
next section. Loss3 can be any reasonable user-given unfair¬ 
ness function. For instance: 
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i=0 



(6) 


Such a Loss3 is free from the division-by-zero problem and 
drives the unit price for offered data usefulness to be the same 
for all participants. Nevertheless, the specific form of Loss3 
has no effect on the theoretical guarantees for objectives other 
than this user-defined fairness. 




Because there is no closed-form solution for the optimal 
h*{-) and we approximate them with neural networks. 
In particular, we construct n neural networks NET^,i G 
N, which share the same set of parameters, to approx¬ 
imate G N, and another n monotonic networks 

(i.e., neural networks with all weights being non-negative) 
NET®,i G N, sharing another set of parameters, to approx¬ 
imate G N. Output nodes of these 2n networks, de¬ 

noted by NET^o, NET®.o, i S N, are combined into a single 
composite neural network in Eigure 2 with the following loss 
function (of which the reasonableness will be proved in the 
next section): 

LOSS = ^ ^'{Ai ReLu[-{S** - S*_\) 

t—0 i—0 

n—1 

- (NET.^o -f NET®.o)] -f A 2 ReLu[^( (7) 

i^O 

[(S'** - S*_\) + (NETf .0 -f NETf.o)] - S**)] 

-f A 3 Loss 3 (t* -f NET.^'.o-f NET®.o,d*)}, 

where T is the sample size. Eor the sample t, r* = 7 *), 

S*_\ = S*(d*_i, 7 iJ, and S*‘ = S*(d*, 7 *). 

The composite neural network can be trained based on par¬ 
ticipants’ behaviors in real scenarios. If such behavioral data 
are not available, EL coordinators may leverage domain ex¬ 
perts’ knowledge to generate synthetic data (d*, 7 *) from the 
estimated prior distribution (A(d), A( 7 )) during the boot¬ 
strapping stage. 

The training procedure for EVCG is described in Algo¬ 
rithm 1. Within this algorithm, CAPITALIZED words refer to 
nodes in the composite neural network, whereas lowercased 
words stand for other types of variables. The value of a node 
is referenced as NODE.uaZue. The composite neural network 
is constructed according to Eq. 7, where values of the follow¬ 
ing variables are fed to the input nodes: 

(d‘, 7 *), r‘, S% S*\iGN,t = 0,...,T-l. ( 8 ) 

Einally, the EVCG payment to participant i is the sum of 


Algorithm 1 Train [NET**], [NET®] for alH e A 

Require: Neural networks: [NETf], [NETf] 

Probability Distribution: A(d) , A( 7 ) 

Ensure: Trained networks: [NET**], [NET®] 

1 : construct the composite neural network for LOSS where 
input nodes are those in Eq. 8 and the LOSS node is 
constructed according to Eq. 7 
2 : PAR ^ list of all parameters of the computational graph 
3: initialize PAR 
4: for I in iange(iterations) do 
5: for f in range(T) do 

6 : randomly draw (d*, 7 *) from A(d), A( 7 ) 

7: compute r*. S'** and all SI* 

8 : end for 

9: assign (d*, 7 *),r*,S**,S**,i S N,t = l,...,Sto 

corresponding input nodes of LOSS. 

10 : feedEorward(LOSS) 

11 : backPropogation(LOSS) 

12: S G- value 

13: PAR.ua/rte ^ PAR.ua/ue — a * S 

14: if r| + NET**’*.o.ua/ite + NET®’*.o.ua/ue < 0 then 

15: increase the bias of the output layer of NET[* by b 

16: end if 

17: end for 


the VCG payment and the optimal adjustment payment: 

K = nid, 7 ) + NET^^o + NETf.o, (9) 

where (d_i, 7 _i) and di are fed into the input nodes of the 
trained networks NET** and NET®, respectively. 

5 Analytical Evaluation 

In this section, we analyze the properties of EVCG. We hrst 
prove for arbitrary hi{d-i,j-i) and increasing gi{di), the 
payment pi(-) = Ti{-) + hi{-) + gi{-) satisfies dominant strat¬ 
egy incentive compatibility and social surplus maximization. 

Proposition 1 (Dominant strategy incentive compatibility). 
For every participant i, truthfully reporting its cost type ji 
and offering its original dataset for FL model training is the 
dominant strategy under FVCG, i.e., 

p,{{d„d_i), (7j,7_i)) - c^{di © ?7*((di,d_i), (7^, 7.^)), 7^) 
>Pi{d,j) - Ci{d^Q'n*{d,'j),-fi), Vdj,d,7i,7. (10) 

Proof. By dehnition of S*{d,'y) (i.e., S*(d, 7 ) = 

max^g[(j i]di„{di)xr.{S(d0J7)-X;”Jo^G(d,077,,7i)},Vd, 7 ), 

we substitute d with {di, d-f) and get 

S*{{di,d_i), (7i,7_j)) > B{{d„d_i) © rf) 

- Ci{di © ry*,7i) - '^Ci{dk © r]k,fk), Vry. (11) 

k^i 

In particular, for rj = {di © ?7*(d,7) 0 di,r]fi{d,j)) G 
[0 i]dim(di)xn, £q holds. Also, because the usefulness 
of the offered dataset (which is verihed in the sandbox) is no 

































greater than the original dataset, di > di and g{di ) > g{di)- 
Therefore, 

S*{{di,d_i), (7i,7_i)) 

>B{{di,d_i) © (di 0 ? 7 *(d, 7 ) 0 dj,r 7 lj(d, 7 ))) 
-'^c,{dkQvl{d,j),%) + g,{d,) 

k^i 

-c^{diQdiQg*{d,j)0d„ji) ( 12 ) 

=S{{d^,d_i) © ?7*(d,7),(7i,7_i)) +5i(d*) 

+ ©(di © g* (d, 7 ), 7 i) - Ci{di © g* (d, 7 ), 7 i) 

=-S'*(d, 7 ) +©(d*) 

+ ©(di © g* (d, 7 ), 7 i) - Q(di © g* (d, 7 ), 7 *). 

Adding hi{d-i,'y-i) — S'^^{d-i, 7 _i) to both sides of Eq. 12 
and substituting Pi (•) = £'*(•) - Sid’) + ©(’) + 
we get 

Pt{{di,d_,),{g„^_i)) - Ci{diQg*{,{d„d_i), ( 7 *,7-»)),7») 

>Pt{d,l) -©(di©r 7 i(d, 7 ), 7 j). (13) 

□ 

Proposition 2 (Social surplus maximization). FVCG maxi¬ 
mizes the social surplus. 

Proof. Suppose d** = argmax^^^{S'(d, 7 )} and S** = 

S{d**, 7 ). We aim to prove that FVCG results in social sur¬ 
plus no less than S**. 

By definition of rj*(d, 7 ), 

S'(d©J 7 *(d, 7 ), 7 ) > S'(d©r 7 , 7 ), Vry. (14) 

Incentive compatibility means d = d, j = j. Therefore, 

S'(d©J 7 *(d, 7 ), 7 ) > S'(d©r 7 , 7 ), Vry. (15) 

Particularly, Eq. 15 holds for g = d** 0 d G [0, l]dim(©)xn^ 
i.e., 

5(d©r7*(d,7),7) > Sided** 0d,j) 

= Sid**,j) = S**. (16) 

The left side of Eq. 16 is the social surplus achieved by 
FVCG, while the right side is the maximum possible social 
surplus for the given (d, 7 ). Therefore, the left side equals 
the right side. □ 

Based on the dominant incentive compatibility guaranteed 
by Proposition 1, we can give conditions for individual ratio¬ 
nality and weak budget balancedness. 

Proposition 3 (Individual rationality). FVCG is individual 
rational (IR) i.f.f. the usefulness of the original datasets and 
the true cost types satisfy 

hdd_„7-0+©(d0 > -[5*(d,7)-^*,(d_„7_,)]. (17) 


Proof. Since playing truthfully is shown to be the dominant 
strategy by Proposition 1, we use d, 7 to substitute d, 7 in 
Eq. 2 and Eq. 4. Then, the utility of participant i becomes 

Mi(d, 7 ) =p,(d, 7 ) - Ciid,eg*{d,-i),gi) (18) 

= S*id,j) - S'*i(d_i,7_i) -f h^id_i,^_f) + giidf). 

IR requires Mi(d, 7 ) > 0, which is equivalent to the inequal¬ 
ity in Eq. 17. □ 

Proposition 4 (Weak budget balance). FVCG is weakly bud¬ 
get balanced (WBB) i.f.f. the usefulness of the original 
datasets and the true cost types satisfy 

n—1 

i^O 

n—1 

< s%dn) - - SUd-in-^)]■ ( 19 ) 

i^O 

Proof. Since playing truthfully is shown to be the dominant 
strategy by Proposition 1, we use d, 7 to substitute d, 7 in 
Eq. 2 and Eq. 4. Then, the total payment to all participants is 

n—1 n—1 

y^p,(d,7) = y^[7(d,7) -f /i*(d_,,7_i) -f 5,(d,)] 
i—0 0 

n—1 

= E][‘5'*(d,7) - 5'*i(d_i,7_i) +©(d*?7-(<^)7).7*) 

+ h^id_i,^_f) + giidf)]. (20) 

WBB means 7) — © 'n*{d,l))- This in¬ 

equality, together with the definition S*{d,j) = Bid e 

'n*id,l)) - J2i=o cidi © p*(d, 7 ), 7 ,), transforms Eq. 20 
toEq. 19. □ 

According to Proposition 3 and 4, in order to find h.*(-) 
and g*i-) that best satisfy IR and WBB, we minimize the 
expected difference between the left side and the right side of 
17 and Eq. 19 when they are violated , i.e., we set Lossl and 
Loss2 as follows: 

n—1 

Lossl = y^ReLu[-( 5 '*(d, 7 ) - S'* j(d_i, 7 _j)) 

1=0 

- (/i*(d_i,7_,)-f pi(d,))], (21) 

n—1 

and Loss2 = ReLu[y^ [(S* (d, 7 ) — S(lj(d_i, 7 _j)) 

i=0 

-f (/i*(d_i, 7 _j) +giid^))] - S*(d, 7 )]. (22) 

Substituting Eq. 21 and 22 into Eq. 5, we get the objective 
function of the composite network in Eq. 7. 

Depending on different assumptions. Loss 1 and Loss2 may 
or may not be reduced to zero by learning the optimal h.*(-) 
and g*i-). Nevertheless, for those h.*(-) and g*i-) that make 
Eq. 17 and 19 hold together for arbitrary (d, 7 ), we have 
Lossl = Loss2 = 0, and hence IR and WBB are satisfied 
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Figure 3: Comparison between different FL payoff schemes 


universally. We can prove that the following inequality is a 
sufficient condition for the existence of such h*{■), g* {■): 

n—l 

Y,{S*{d,'y) - SU{d-^n-^)] < S*{d,'y), Vd, 7 , (23) 

i=0 

where S'* 7 _i) is the maximum social surplus attained 

by the n — l participants except for i. 

The following two assumptions lead to Eq. 23, thus pro¬ 
viding guarantees for IR and WBB: 1) the federation revenue 
function is super additive (i.e., B[d) > Er=o ^K),vd) 
and 2 ) the marginal effect of the usefulness of one partici¬ 
pant’s data decreases as the usefulness of other participants’ 
data increases {i.t., B{di,d-i) — B{d\^d-i) < B{di,d'_^) — 

B{d'^, d'_^),ydi > d[,d_i > d'_^ ). These two assumptions 
apply to most FL application scenarios. Proofs of Eq. 23 
and its relationship with the aforementioned assumptions are 
beyond the scope of this paper and hence are omitted here. 

6 Experimental Evaluation 

In this section, we experimentally evaluate the performance 
of FVCG against five existing payoff schemes. They are; 

1. Shapley: the federation revenue is shared among partic¬ 
ipants according to the Shapley value [Jia et ai, 2019]; 

2. Union: participant i’s share of the federation revenue 
follows the Labour Union game [Gollapudi et al, 2017] 
payoff scheme and is proportional to its marginal con¬ 
tribution to the revenue of the federation formed by its 
predecessors, i.e., B{{do, ■ ■ ■, di)) — B{{do ,..., 

3. Individual: participant Ts share of the federation rev¬ 
enue is proportional to its marginal contribution to the 
federation revenue [Yang et al., 2017]; 

4. Linear: participant i’s share of the federation revenue is 
proportional to the usefulness of its contributed data (this 
is a payoff scheme that we designed for experimental 
comparison purposes only); and 

5. Equal: the federation revenue is equally divided among 
participants in this federation [Yang et ai, 2017]. 

We built a simulator which creates participants with diverse 
characteristics to study how the different payoff schemes per¬ 
form. The relative performance of the payoff schemes in the 
simulation is more important than the exact values. 


6.1 Experiment Settings 

In the experiment, we carry out simulations based on hypo¬ 
thetical revenue and cost functions as follows: 


B(d) = 


\ 


n—l 

^ndCi{di,J^) = -fidi.i G N. (24) 

2=0 


The user-dehned unfairness function is set to be that in Eq. 6 . 
We set n = 10 and A(di), A( 7 j), the prior belief on useful¬ 
ness and cost types, to be one-dimensional uniform distribu¬ 
tions on [0, 5] and [0,1], respectively. We set the the hyper¬ 
parameters in Algorithm 1 to be Ai = 0.3, A 2 = 0.3, A 3 = 
0.4, T = 100, a = 0.001, b = 1.0, and NETf, NET® to have 
three 10-dimensional hidden layers and one 50-dimensional 
hidden layer, respectively. 

We run simulations to compare the FVCG mechanism with 
other payoff schemes. We simulate 1,000 samples for 10 par¬ 
ticipants. In each sample t, we randomly generate the useful¬ 
ness dl of the original dataset and the true cost type 7* from 
A{di), A{'yi). For each payoff scheme, we assume at the be¬ 
ginning, each participant contributes (or offers) k = 10 % 
of its data. In each iteration s, based on other participants’ 
behaviors, each participant choose between 1 ) to stop offer¬ 
ing/contributing data; 2 ) to offer/contribute another 10 % of its 
data; 3) to withdraw 10% of its data from the federation; or 
4) to maintain the status quo. The iteration stops when equi¬ 
librium is reached or after 1,000 iterations. For FVCG, we 
further assume each participant randomly reports a false cost 
type at hrst and adjust the reported cost type in each iteration. 

After the simulations reach the final state, we calculate the 
average of the following measures across all participants and 
all samples; 

1. Ojfer Rate: the ratio between the usefulness of the of¬ 
fered dataset to that of the original dataset, d\/d\ (for 
payoff schemes other than FVCG, d\ = d*); 

2. Quit Rate: (No. of participants that quit the federation) / 
(No. of total participants); 

3. Federation Revenue: B{d*'); 

4. Social Surplus: S'(d*, 7 ‘) = B{S) — 

which is the total utilities of all participants (including 
the FL coordinator); and 

5. Budget Surplus: B{S) — p^, which is also the 

proht gained by the FL coordinator. 




6.2 Results and Discussion 

Experimental evaluation results (Figure 3) are consistent with 
our theoretical predictions. The FVCG mechanism incen- 
tivizes participants to offer all their data (i.e., the offer rate 
is 100%) and keeps all participants in the federation (i.e., the 
quit rate is 0). Although the federation revenue of FVCG 
is lower than other payoff schemes (because FVCG rejects 
data if the marginal contribution cannot cover the marginal 
cost), FVCG results in the highest social surplus and signifi¬ 
cant budget surplus (around 15% of the federation revenue). 
This means the federation as a whole has the highest excess 
economic gain over costs, and the FF coordinator can make 
considerable profits by adjusting FF participation. 

7 Conclusions and Future Work 

In this paper, we presented the FVCG mechanism for feder¬ 
ated learning. It incentivizes participants to join FF model 
training with their original dataset and truthfully report their 
cost types. It maximizes social surplus, minimizes ex¬ 
pected deviations from individual rationality, is by expecta¬ 
tion weakly budget balanced, and supports user-defined fair¬ 
ness criteria. With the theoretical guarantee of incentive com¬ 
patibility, FVCG achieves a desirable trade-off among all 
these considerations. 

In subsequent research, we will further study how to evalu¬ 
ate the usefulness of datasets, identify the federation revenue 
function, and estimate cost types in a privacy-preserving man¬ 
ner. Each of them is a promising research direction. 
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